Allow argument-provision checks of templated fields in operator __init__ - #70505
Merged
Conversation
shahar1
requested review from
amoghrajesh,
ashb,
bugraoz93,
gopidesupavan,
jason810496,
jscheffl and
potiuk
as code owners
July 27, 2026 08:42
shahar1
force-pushed
the
validate-init-provision-checks
branch
from
July 27, 2026 08:58
adfa421 to
0026dec
Compare
This was referenced Jul 27, 2026
This was referenced Jul 27, 2026
1 task
mitre88
added a commit
to mitre88/airflow
that referenced
this pull request
Jul 27, 2026
The provision checks in GCSDeleteObjectsOperator are false positives under the relaxed validator definition proposed in apache#70505, so they stay in the constructor and the class keeps its exemption entry.
This was referenced Jul 27, 2026
vincbeck
approved these changes
Jul 27, 2026
The rendering pipeline is one-way with respect to None: an unset field stays unset, but a field that was passed can render to None. A check that only asks which arguments the author supplied therefore has to run in the constructor, and the hook now permits it while still rejecting every read of the un-rendered value.
shahar1
force-pushed
the
validate-init-provision-checks
branch
from
July 27, 2026 21:32
0026dec to
b3e2c33
Compare
The rule tells contributors to write provision checks as `is None`, but for a required argument that rewrite starts accepting the empty string the old truthiness check rejected, silently weakening the operator.
Contributor
Backport failed to create: v3-3-test. View the failure log Run detailsNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
You can attempt to backport this manually by running: cherry_picker 83968fb v3-3-testThis should apply the commit to the v3-3-test branch and leave the commit in conflict state marking After you have resolved the conflicts, you can continue the backport process by running: cherry_picker --continueIf you don't have cherry-picker installed, see the installation guide. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
related: #70296, #70347, #70297
Human Summary
As it turns out, the check I've introduced in #70297 might have been too strict, and we should allow some specific checks to still run in the constructor. This PR relaxes the requirements and removes false positives (according to the new definition) from the exemptions list.
Created a separate issue (#70503) to track reverts.
AI Summary
Click here
validate-operators-initflags every read of a template-field parameter in__init__. That is right for reads of the value, but wrong for reads of provision — "did the author pass this argument?" — which is what mutually-exclusive-argument checks ask.The constructor is in fact the only place that can answer it. With
render_template_as_native_obj=Truea provided field renders toNone, so the same check inexecute()reports a supplied argument as missing. Forcing the move also turns a static authoring mistake into a per-task-instance runtime failure, and — because a rendered value can legitimately be falsy — invites truthiness tests that silently accept conflicting arguments.This adds one sanctioned pattern:
field is None/field is not Noneon a template-field parameter (orself.<field>) is allowed in__init__. Everything else — truthiness,in,.startswith,isinstance, transformations — stays flagged.Measured against the burn-down. Replaying both checkers over every exemption entry ever removed (31 PRs, 48 entries): 8 entries across 5 PRs would have needed no operator code change at all — #70341, #70338, #70348, #70392, #70326 moved pure
is Noneexclusivity checks intoexecute()verbatim. Two cost more than churn: #70348 broke the Rendered Templates view and needed follow-up #70373, and #70326 made a released operator argument required. Five currently-listed entries clear with no code change and are removed here; two of them are already claimed by contributors on #70296.That replay counted only reverts that are free — entries whose findings drop to zero with no edit to the operator. It says nothing about provision checks written as truthiness (
if a and b: raise), which score identically under both checkers and so are invisible to it; those needed a rewrite to stay in__init__, not just the narrowed rule. A separate pass over guard shape found 10 such guards across 6 PRs; the 4 mutually-exclusive ones are tracked on #70503, and the doc now warns that rewriting a required-argument check tois Noneloosens it.The remaining 40 entries stay flagged, correctly — they are genuine value reads (
ruleset.strip(),isinstance(mongo_query, list),source not in SUPPORTED_SOURCES).Second half of the change: nested statements are now checked for invalid template-field assignments. Sanctioning the
is Noneread would otherwise letif field is None: self.field = derive(other)through, since the assignment check only looked at top-level statements. A nested assignment still cannot satisfy the requirement that the field be assigned — it may not run — and nesteddef/classscopes are skipped. That also closes a pre-existing hole with an unrelated guard.Testing Done
scripts/tests/ci/prek/test_validate_operators_init.py: 35 passed. 8 of the 10 new/changed cases fail against the pre-change checker; the 2 that don't are guards against over-sanctioning (truthiness andfoo.get('x') is Nonemust stay flagged).files:regex: exit 0, no stale exemptions.prek run --from-ref main --stage pre-commit: clean (includes mypy for scripts).Reviewer note: reconciling merged precedent
Five merged burn-down PRs moved pure provision checks into
execute()and left comments and regression tests asserting that placement. Nothing breaks — the new rule permitsis Nonein__init__, it does not require it — but those files now demonstrate the pattern the docs discourage, and contributors on #70296 learn the rule by reading them. Reverting them is tracked separately so this PR stays reviewable.No newsfragment: prek hook, contributing docs, and a clarification to an existing limitations section. Not user-facing behaviour.
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 5) following the guidelines